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This  paper  discusses  automatic  detection  and  exploitation 
of  structural  redundancy  in  large-scale  mathematical  programming 
models.   From  our  perspective,  such  redundancy  represents  embedded 
special  structure  which  can  give  significant  insight  to  the  model 
proponent  as  well  as  greatly  reduce  solution  effort.   We  report 
experiments  with  real-life  linear  programming  (LP)  and  mixed- 
integer  (MIP)  models  in  which  various  methods  are  developed  and 
tested  as  integral  modules  in  an  optimization  system  of  advanced 
design.   We  seek  to  understand  the  modelling  implications  of  these 
embedded  redundancies  as  well  as  to  exploit  them  during  actual 
optimization.   The  latter  goal  places  heavy  emphasis  on  efficient, 
as  well  as  effective,  identification  techniques  for  economic  appli- 
cation to  large  models.   Several  (polynomially  bounded)  heuristic 
detection  algorithms  are  presented  from  our  work.   In  addition, 
bounds  are  reported  for  the  maximum  row  dimension  of  the  more 
complex  structures.   These  bounds  are  useful  for  objectively 
estimating  the  quality  of  heuristically  derived  assessments  of 
structural  redundancy.   Finally,  some  additional  suggestions  are 
made  for  analyzing  nonlinear  programming  (NLP)  models. 


1.   INTRODUCTION 

Automatic  detection  and  exploitation  of  structural 
redundancy  in  large-scale  linear  programming  (LP)  (as  well  as 
mixed  integer  programming  (MIP)  and  nonlinear)  models  has  been 
one  subject  of  a  continuing  research  program  conducted  at  the 
Naval  Postgraduate  School  and  UCLA  over  the  past  decade.   This 
exposition  draws  from  various  results  in  that  effort,  and  refers 
(sparingly)  to  significant  work  by  other  researchers.   The  refer- 
ences contain  complete  descriptions  of  these  results  for  the 
interested  reader. 

Our  scope  is  intentionally  limited  to  automated  methods 
of  sufficient  efficiency  to  enable  us  to  economically  apply  them 
to  real-world  optimization  problems.   Thus,  we  consider  only 
those  approaches  showing  greatest  promise  for  immediate  practical 
application.   Although  the  interpretations  of  embedded  model 
redundancy  can  lend  profound  insights  in  their  own  right,  we  are 
equally  interested  in  detecting  errors  in  data  preparation  and 
model  generation—mathematically  mundane  issues  of  fundamental 
importance  to  the  practitioner. 

In  this  context,  our  definition  of  structural  redundancy 
includes  not  only  features  which  permit  reduction  of  effective 
problem  size,  but  also  those  embedded  special  structures  which 
invite  application  of  special  solution  methods  with  enhanced 
efficiency.   This  somewhat  expanded  view  of  redundancy  admits 
features  which  yield  to,  for  instance,  basis  factorization  or 
decomposition . 


The  sheer  size  of  contemporary  large-scale  LP  models 
presents  significant  computational  difficulties,  even  for  dis- 
covery of  otherwise  elementary  structures  (in  the  sense  of  formal 
complexity) .   Implementation  of  effective  structural  analysis 
procedures  is  primarily  a  matter  of  exercising  large-scale  data 
structures  efficiently.   As  we  shall  see,  though,  these  practical 
considerations  can  give  significant  theoretical  guidance  in  the 
specification  of  efficiently  achievable  classes  of  model  trans- 
formations. 

That  detection  of  embedded  special  structure  can  be  of 
practical  importance  in  actual  model  solution  is  undisputed.   It 
is  widely  known  that  explicit  simplex  operations  can  be  materially 
improved  in  efficiency  by  incorporation  of  basis  factorization 
methods  (e.g.  Graves  and  McBride  [12,  15]).   The  details  of 
such  modifications  of  the  simplex  procedure  are  not  given  here. 
However,  the  underlying  themes  of  simplex  factorization  are  the 
substitution  of  logic  for  floating  point  arithmetic,  and  separa- 
tion of  the  apparent  problem  monolith  into  more  manageable 
components. 

This  work  deals  primarily  with  row  factorizations.   The 
pervasive  implied  problem  for  row  factorization  is  the  identifi- 
cation of  the  best   embedded  structure  from  all  those  that  may  lie 
at  hand  in  any  particular  model.   Conventional  wisdom  differs  as 
to   the  criterion  for  this  discrimination  among  factorizations 
of  the  same  class.   However,  it  is  generally  accepted  that  the 
row  dimensionality  of  the  factorization  serves  as  an  excellent 
measure  of  effectiveness.   In  this  sense,  embedded  special 


structures  fall  naturally  into  a  taxonomy  implied  by  the  intrinsic 
complexity  of  the  associated  maximum  row  identification  problems. 
We  proceed  with  a  discussion  of  several  types  of  embedded 
special  structures  detectable  by  efficient  polynomially  bounded 
algorithms.   These  structures  are  considered  in  increasing  order 
of  maximum  row  identification  complexity.   We  emphasize  that 
efficient   polynomial  algorithms  are  operationally  defined  here 
as  low-order  polynomial  in  terms  of  intrinsic  problem  dimensions 
(e.g.  number  of  rows,  columns/  and  non-zero  elements),  and  not 
in  terms  of  the  total  volume  of  model  information  (e.g.  total 
number  of  bits  in  all  coefficients,  ad  nauseam). 


2.   OVERVIEW  OF  THE  ANALYSIS 

We  are  usually  faced  with  the  following  practical  situation. 
An  LP  (or  MIP)  is  presented,  typically  in  MPS  format  on  magnetic 
tape.   Some  documentation  and  possibly  solution  experience  is 
also  provided,  but  the  matrix  generator  and  source  data  elements 
for  the  model  are  rarely  at  hand.   The  model  is  frequently  sent 
to  us  because  of  some  difficulty  encountered  in  preliminary  solu- 
tion attempts. 

Our  efforts  are  devoted  to  two  issues:   Analysis  of  the 
LP,  and  solving  it  efficiently. 

The  analysis  is  initially  focused  on  rather  ordinary 
details,  primarily  establishing  computer  compatibility,  and  on 
review  of  gross  problem  statistics  such  as  those  shown  in  Table  1 
for  the  sample  problems  we  will  discuss  further.   At  this  stage 
we  attempt  to  detect  outright  disasters  before  investing  more 
time  and  money. 

For  instance,  we  have  frequently  found  that  the  actual  LP 
bears  little  resemblance  to  the  intended  formulation  due  to  matrix 
generator  difficulties,  human  error,  or  data  base  failures.   Even 
an  LP  which  seems  to  follow  its  formulation  template  may  exhibit 
superfluous  objective  functions,  right  hand  sides,  and  so  forth. 

After  the  cursory  review,  a  representative  model  is  selected, 
set  up  and  input  to  an  internal  data  structure  for  detailed  analy- 
sis.  The  input  process  is  relatively  expensive  for  large  LP  models, 
involving  conversion  of  thousands  of  records.   Row  and  column 
summary  statistics  are  produced  and  reviewed.   At  this  point, 
pathological  coefficient  scaling  is  frequently  revealed;  this  is 
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sometimes  caused  by  poor  modeling,  and  occasionally  by  bad  source 
data  elements.   For  example,  we  have  found  at  this  point  that 
unwanted  model  features  (such  as,  say,  production  capacity  con- 
straints) have  been  "relaxed"  in  an  ad  hoc  fashion  by  providing 
outrageous  dummy  coefficients  (e.g.  essentially  infinite  capacity 
or  infinitesimal  production  rates).   These  redundancies,  while 
innocently  intended,  can  play  havoc  with  LP  solution  procedures 
(this  is  especially  true  for  MIP  and  nonlinear  algorithms) . 

Next,  we  apply  a  set  of  simple    reductions    to  the  LP  model. 
At  this  stage,  we  identify  redundancies  with  two  goals.   First, 
we  want  to  complete  our  "bottom-up"  analysis  of  the  model.   How- 
ever, we  also  seek  to  set  the  stage  for  actual  model  solution. 

In  this  sense,  we  cannot  (ordinarily)  afford  and  we  do  not 
(routinely)  apply  at  large  scale  reduction  methods  which  are  com- 
putationally equivalent  to  actual  solution  of  the  LP.  Rather,  we 
discover  as  much  as  possible  about  redundancy  in  the  model  by 
efficient,  polynomially  bounded  static  analysis  not  employing 
basis  exchanges.  From  this  we  attempt  to  infer  qualitatively  the 
complete  redundancy  structure  of  the  model  at  hand. 

Also,  we  ignore  structural  features  which  have  little  or 
no  bearing  on  our  analysis  or  solution  effort.   For  instance,  we 
are  indifferent  about  degeneracy,  since  our  solution  procedures 
exploit  this  delightful  property  (Graves  [11  ] )  . 

Similarly,  we  do  seek  certain  properties  which  may  make 
our  solution  procedures  much  more  efficient,  but  which  may  not 
be  so  attractive  to  analysts  using  other  methods.  Among  these 
properties  are  embedded  generalized  upper  bounds  (GUB)  and  net- 


work,  or  pure  network  (NET)  rows,  for  which  our  interest  is 
particularly  keen  (Bradley,  Brown  and  Graves  [6],  Brown  and 
Graves  [8],  Graves  and  Van  Roy  [13]). 

Actual  solution  of  the  model  follows  these  analysis  efforts 
unless  we  find  that  the  model  requires  significant  modification 
or  managerial  review.   For  models  that  merit  solution,  we  see  no 
reason  to  impose  unreasonable  restrictions  on  the  model  builder. 
That  is,  the  responsibility  for  efficient  solution  is  ours  regard- 
less of  the  redundancy  structure  found,  as  long  as  this  redundancy 
is  not  in  conflict  with  the  intent  of  the  modeler.   Thus,  all 
reductions  must  provide  an  equivalent  solution  to  the  model  as 
originally  posed,  and  can  not  require  that  the  model  be  changed 
or  severely  modified  externally  (e.g.  requiring  general  linear 
transformation  of  the  formulation  to  suit  the  solver  is  out  of 
the  question) . 


3.   DETAILS  OF  THE  ANALYSIS 

For  analysis,  the  linear  program  is  stored  in  a  sparse 
data  structure.   Nonzero  coefficients  are  stored  along  with  the 
corresponding  row  index  by  column  with  access  via  column  entry 
points.   Each  row  and  column  has  associated  with  it  an  external 
label,  several  coefficient  values  representing  upper  and  lower 
bounds  and  ranges,  and  a  coded  tag  giving  the  type  of  constraint 
(e.g.  equation)  or  column  and  its  status  in  the  analysis. 

Although  conversion  to  a  super-sparse  data  structure  is 
subsequently  required  for  the  solution  of  the  model,  the  sparse 
structure  is  much  more  convenient  for  analysis  of  the  model 
composition  and  for  model  modifications.   Very  little  auxiliary 
storage  is  required  for  the  analysis  and  practical  problems  at 
large  scale  are  routinely  analyzed.   The  system  is  designed  to 
operate  on  problems  with  up  to  30,000  rows  plus  columns.   (E.g. 
see  Bradley,  Brown  and  Galatas  [4]  for  FORTRAN  examples  using 
these  data  structures.) 

Our  analysis  is  confined  to  reductions  that  do  not  change 
the  feasible  region.   The  analysis  can  also  be  called  "orthogonal" 
in  that  the  reduction  tests  are  made  on  the  current  problem  with 
no  pivotal  transformations  actually  performed.   The  reductions 
may  show  how  to  transform  the  problem  by  removing  columns  and 
constraints  and  by  the  elimination  of  columns  (equivalent  to 
pivoting) ,  but  the  tests  are  applied  only  to  the  current  repre- 
sentation of  the  problem. 

The  analysis  is  applied  to  a  fully  ranged  and  bounded 
linear  program. 


min    T . c . X . 
L3    3    3 


s.t.    r.  <  V.a. .X.  <  r   V  i    (ranged  constraints) 


£.  <  X.  <  u.   V  j    (simple  bounds)  . 

n    :    :    J 


Some  ranges  and  bounds  may  be  missing  (that  is,  +ca     or   -OT) 


3.1.   Simple  Reductions 

Singleton   column.       If  a  column  has  a  single  nonzero 
coefficient/  the  column  can  be  removed.   The  ranges  of  the  con- 
straint that  contain  the  nonzero  coefficient  are  modified  to 
construct  an  equivalent  problem. 

Fixed  column.       If  a  column  has  been  fixed  at  a  certain 
value,  or  (equivalently)  its  upper  and  lower  bounds  are  equal, 
it  can  be  removed.   The  ranges  of  all  constraints  that  have  a 
nonzero  coefficient  in  the  removed  column  are  modified. 

Vacuous    columns.      Columns  with  no  nonzero  coefficients 
may  be  removed.   The  associated  variable  may  assume  any  value 
between  its  upper  and  lower  bounds. 

Inconsistent    column.      Any  column  with  its  upper  bound 
strictly  less  than  its  lower  bound  indicates  that  the  linear 
program  has  no  feasible  solution. 

Free   column.      A  free  column  results  from  modeling  a 
variable  that  has  no  upper  or  lower  bound  or  by  analysis  that 
can  show  that  neither  the  upper  nor  lower  bound  is  necessary  to 
define  the  feasible  region. 

Singleton   constraint.       If  a  constraint  has  a  single 
nonzero  coefficient,  the  constraint  may  be  removed.   The  bounds 
of  the  column  that  contains  the  coefficient  are  modified.   Simple 
bounds  are  often  revealed  this  way  when  they  are  inadvertently 
expressed  as  explicit  constraints. 


10 


Redundant   constraints .      A  constraint  is  redundant  if 
its  removal  does  not  change  the  feasible  region.   An  examination 
of  the  bounds  on  columns  with  nonzero  coefficients  yields  a  test 
for  a  redundant  constraint.   For  constraint   i   define 

R1=    T   a..u.+    I      a .  . I .    , 

a.  >0  ^  =>    a.  <0  ^  3 

ID  13 

and 

R.  =    V   a.  .1 .  +    y   a.  .u.  . 

1    a. So  ^  ^    a. So  ^  ^ 
iD  13 

A  constraint  is  redundant  if   R   5  r   and   R.  >  r. .   If  only  one 

11         2 

inequality  holds,  the  corresponding  range  can  be  eliminated  (that 
is,  set  to   °°  or   -«)  . 

Constraints    that   fix   variables .       If   R   =  r.   or 

J  1 

R.  =  r   then  each  column  with  a  nonzero  coefficient  in  the  con- 

1 

straint  must  be  fixed  at  the  appropriate  bound  in  order  for  the 
constraint  to  be  satisfied.   The  constraint  can  then  be  removed. 

Inconsistent    constraints .       If   R   <  r.   or   R.  >  r  , 
then  the  LP  has  no  feasible  solution. 

Vacuous   constraints .   Constraints  with  no  nonzero 
coefficients  may  be  removed.   If   r   <  0   or   r .  >  0 ,   then  the 
LP  has  no  feasible  solution. 

All  of  the  tests  above  are  applied  in  a  single  pass. 
Since  the  reductions  (if  any)  may  make  it  possible  to  identify 
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new  reductions,  the  complete  analysis  consists  of  repeated  passes 
until  no  additional  reductions  are  found.   Table  2  displays  the 
results  for  the  sample  problems.   The  times  given  are  for  execution 
on  an  IBM  360/6  7  using  FORTRAN  H  (Extended)  without  code  optimi- 
zation. 

With  real-life  LP  (and  MIP)  models,  a  remarkably  large 
fraction  of  model  constraints  can  be  removed  by  these  simple 
techniques.   For  some  cases,  models  have  been  nearly  solved   this 
way. 

We  have  often  been  surprised  at  the  number  of  reduc- 
tions achieved  after  several  passes.   The  repeated  passes  can 
serve  to  unravel  a  model  and  reveal  special  structure  that  is 
quite  obscure  in  a  static  analysis. 

Experiments  with  some  of  these  reductions  have  been 
reported  by  Brearley,  Mitra  and  Williams  [7].   More  extensive 
work  at  large  scale  has  been  done  by  Bradley,  Brown  and  Graves 
[5]  and  by  Krabek  [14] . 
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3.2.   Transformation  Reductions 

Equations  confine  the  feasible  region  to  an  affine 
subspace  of  lower  dimension.   It  is  possible  to  eliminate  an 
equation  and  a  column  with  a  nonzero  coefficient  in  the  equation. 
If   a.,  ¥■   0   in  an  equation  constraint   i,   then 


X 


=  (r1-  I      a   X  ,)/a 


can  be  used  to  eliminate   X,   from  the  problem.   The  equation  is 
eliminated  but  the  bounds  on   X,   generate  a  constraint 


\  -  (ri-  I.  aijxj)/aik  -  \  • 


This  elimination  is  equivalent  to  pivoting  on   a-k«   Although 

repeated  application  will  remove  all  equations,  this  will  not  in 

general  make  the  problem  easier  to  solve  or  reduce  the  number  of 

constraints. 

There  are  two  special  cases  where  the  elimination  of  a 

column  can  reduce  the  number  of  constraints.   If  the  column  X, 

k 

has  no  upper  or  lower  bound  (i.e.  a  "free"  variable),  then  the 
generated  constraint  is  redundant  and  thus  may  be  eliminated. 

If  the  equation  has  only  two  columns  with  nonzero 
coefficients,  the  generated  constraint  is  a  simple  bound  on  a 
variable  and  thus  may  be  combined  with  the  existing  bounds  on 
the  variable. 

A  particular  type  of  constraint  that  is  common  to  many 
models  leads  to  an  equation  that  has  a  nonzero  coefficient  for 
a  column  with  no  bounds.   Commonly  called  material   balance    equations  3 
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these  constraints  set  one  nonnegativa    column  (i.e.  with  infinite 
upper  bound)  equal  to  the  sum  of  several  other  nonnegative  columns. 

x  -  I     X  =  0  , 

0  5  Xk,   0  <  X.,   j  €  J  . 

Since  the   X .   j  €  J   are  all  restricted  to  be  nonnega- 
tive, it  is  easy  to  see  that  the  bound  X,  >  0   is  redundant — that 
is,  if  it  is  eliminated  the  feasible  region  is  unchanged.   Thus 
k   can  be  regarded  as  a  column  with  no  bounds  and  the  elimination 
of   X,   will  reduce  the  number  of  constraints  by  one. 

Notice  that  although  the  analysis  of  the  material  bal- 
ance equation  results  in  removing  the  bound  from  X,  ,   when 
eliminating  X,   any  single  equation  that  has  a  nonzero  coeffi- 
cient in  column  k   can  be  substituted  out. 

The  analysis  applies  to  a  generalization  of  material 
balance  rows: 

a . ,  X.  +   T  a .  . X  .  =  b .  , 

ik  k    .Lj   i;,  2  i 

0  <  X,  ,  I  .    <  X .  5  u .  , 
k'    3  j  3 

where   a..  >  0,   b.   and  I.    >  0   and   a.  .  <  0  V  j€J.   Although 
ik        i        3  2.3  J  ^ 

this  is  not  the  most  general  form  which  can  be  used  to  designate 
that  the  bound (s)  on   X,   are  redundant,  this  form  (and  its  nega- 
tive) captures  all  the  cases  in  the  real- world  problems  that  we 
have  analyzed. 
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Doubleton   equations .   Equation  constraints  with  exactly 
two  nonzero  coefficients  can  be  identified  for  immediate  elimina- 
tion.  The  elimination  of  the  equation  and  one  of  two  columns  is 
accomplished  as  described  above,  the  generated  inequality  contains 
just  a  single  nonzero  coefficient  so  it  can  be  removed  by  modify- 
ing the  bounds  on  the  surviving  column. 

Free    column   equations .      An  equation  with  a  nonzero 
coefficient  for  a  free  column  can  be  identified  for  immediate 
elimination  along  with  the  free  column.   The  transformation  to 
remove  the  constraint  and  column  is  equivalent  to  pivoting  on 
the  nonzero  coefficient. 

Redundant   bounds.      Our  analysis  uses  only  the  generalized 

material  balance  equation  to  identify  columns  that  have  no  bounds. 

It  is  possible  to  generalize  this  idea  to  include  the  use  of  all 

constraints  to  identify  such  columns.   IJach  constraint  may  be  used 

to  generate  bounds  on  all  the  columns  with  nonzero  coefficients. 

For  constraint   i  with   a..  >  0 

lk 


I     a .  . u .  -   V   a.  .1.   +  r .  <  a . ,  X, 

a.  So  1D  3    a.<0  ^    3     x    lk  k 
ID  ID 

<  r1  -   I      a.  .£.  -   I      a..u.  . 

a. So  1:  3        a. So  ^  3 
ID  ID 

D^k 

An  analogous  result  can  be  constructed  for   a.,  <  0. 

lk 

The  intersection  of  these  bounds  from  all  the  constraints 
infer  bounds  for  column  k  which  may  reveal  I,       and   u,   to  be 
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redundant  and  thus  permit  k   to  be  designated  a  free  column, 
or  which  can  be  used  to  tighten  I,       and   u,  . 

Although  this  test  can  be  done  efficiently  for  any- 
particular  a .,  ,   there  are  potentially  quite  a  large  number  of 
candidates  to  test.   We  have  not  included  this  test  in  our  results 
here,  and  we  further  believe  that  there  are  few  real  LP  problems 
for  which  this  test  yields  significantly  more  columns  without 
bounds  than  examination  for  generalized  material  balance  equations. 

One  particular  situation  for  which  this  test  may  be 
selectively  applied  is  that  in  which  a  coefficient  is  much  larger, 
or  much  smaller  than  its  cohorts.   In  this  case  the  inferred  bounds 
may  reveal  infinitesimal  or  (respectively)  gigantic  bounds  for 
X,  ,   possibly  suggesting  deletion  of  the  column. 

Table  2  shows  doubleton  equations  found  after  the  fina}. 
simple  reduction  pass.   Note  the  last  model  (ODSAS)  for  which 
almost  all  constraints  are  identified  as  doubleton  equations. 

A  sample  analysis  for  material  balance  equations  per- 
formed on  the  PAPER  model  detected  1,645  such  constraints. 

It  is  not  always  obvious  whether  actually  applying  a 
particular  transformation  reduction  will  produce  an  LP  model 
which  is  easier  to  solve.   In  particular,  transformation  reduc- 
tions can  generate  a  "reduced,  equivalent  LP"  which  is  actually 
denser,  and  not  necessarily  as  well-scaled  as  its  progenitor. 

On  the  other  hand,  some  reductions  offer  a  decided 

advantage  for  solution  efficiency.   For  constraints  like 

X .  -  X.  =0,   X.,X.  >  0,   both  variables  must  be  in  the  basis  for 
j        k        D   k 

them  to  assume  a  positive  value.   For  many  commercial  linear 
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programming  systems,  partial  pricing  and  the  lack  of  effective 
mechanisms  to  cope  with  degenercy  do  not  allow  the  efficient 
treatment  of  the  special  relationship  between  the  variables. 
Similarly,  for  material  balance  equations  with  nonnegative  vari- 
ables, at  least  two  variables  with  coefficients  of  opposite  sign 
must  be  in  the  basis  before  any  of  the  variables  can  assume  a 
positive  value.   The  transformation  reductions  eliminate  these 
particular  instances  where  relationships  among  variables  inter- 
fere with  the  solution  progress. 

For  large-scale  models,  we  analyze  the  reductions  care- 
fully, using  all    information  available  for  the  model  and  the 
problem  it  addresses.   Numerical  and  structural  consequences  of 
reductions  are  critically  reviewed  in  concert  with  the  algebraic 
interpretations  and  modeling  discoveries  which  they  characterize. 

Notice  that  both  simple  and  transformation  reductions 
may  be  viewed  as  linear  operators  that  do  not  change  the  feasible 
region  of  the  problem.   After  the  reduced  problem  is  solved,  the 
inverse  operators  applied  to  the  optimal  solution  construct  an 
optimal  solution  to  the  original  problem. 

The  analysis  makes  no  special  use  of  an  objective  func- 
tion.  Nonbinding  constraints  (that  is,  "free  rows")  may  be 
included  in  the  problem.   The  objective  function  and  any  free  rows 
designated  to  be  included  are  only  examined  in  determining  if  a 
column  has  a  single  nonzero  coefficient.   Thus  the  reduction  can 
be  done  for  several  different  objective  functions  simultaneously. 
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3. 3.   Generalized  Upper  Bounds 

Rows  for  which  each  column  has  at  most  one  non-zero 
coefficient  (restricted  to  those  rows)  collectively  form  a  gen- 
eralized upper  bound  (GUB)  set.   Usually,  we  additionally  require 
that  the  coefficients  in  these  rows  be  capable  of  being  rendered 
to   ±1  by  simple  row  or  column  scaling. 

The  problem  of  identifying  a  GUB  set  of  maximum   row 
dimension  is  NP-hard,  making  optimal  GUB  identification  algo- 
rithms hopelessly  inefficient  for  our  purposes.   Heuristics 
adapted  from  work  by  Graves  and  by  Senju  and  Toyoda  [13]  (see  also 
Brearley,  Mitra  and  Williams  [7])  work  very  effectively  and  depend- 
ably at  large-scale  to  find  maximal    GUB  sets. 

Unfortunately,  the  problem  of  determining  just  the  size 
of  the  maximum  GUB  set  is  also  NP-hard.   However,  Brown  and  Thomen 
[9]  have  developed  bounds  on  the  size  of  the  maximum  GUB  set  which 
are  sharp  and  easily  computed.   These  bounds  have  been  used  to 
show,  in  some  cases,  that  maximum  GUB  sets  have  been  achieved  via 
heuristic  methods.   In  any  case,  the  bounds  provide  an  excellent 
objective  measure  of  the  quality  of  any  GUB  set,  regardless  of 
the  means  of  its  derivation.   Frequently,  manual  GUB  analysis 
will  suffice  for  models  with  amenable  structure. 

The  bounds  are  developed  in  terms  of  the  number  of  dis- 
tinct conflicts   present  in  the  model.   Two  rows  are  in  conflict 
if  they  each  have  a  non-zero  element  in  a  common  column,  making 
them  mutually  exclusive  in  a  GUB  set.   If   s.   is  the  number  of 
rows  in  conflict  with  row  i,  then  the  total  problem  conflict 
count  for  a  model  with   m  rows  is 
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c  =  j  I    si  <  j  m(m-l)  . 


A  problem- independent  bound  on  the  size  of  the  maximum 
GUB  set  is 


u,  =  I   .  5  +  / .  2  5  +  m  ( m- 1 )  -2c  , 

where     indicates  truncation  to  the  next  lower  integer, 
A  tighter,  problem-dependent  bound  is 


m-f|  i  c  -  (m-y)y 


u2  = 


.5  +  /.25  +  y (2m-y-l)  -  2c  ,      c  >  (m-y) y  ; 


where 


y  =  max  s .  / 
i 

and     indicates  rounding  to  the  next  higher  integer. 

Tighter  upper  bounds  have  been  derived  for  the  size  of 
the  maximum  GUB  set,  as  well  as  lower  bounds. 

Table  3  contains  the  results  of  automatic  GUB  identifica- 
tion applied  to  the  benchmark  models.   Constraint  eligibility  is 
based  on  the  capability  to  scale  the  row  to  contain  only   0,  ±1 
coefficients.  GUB   quality    is  the  number  of  GUB  rows  found,  ex- 
pressed as  a  percentage  of  the  best  known  upper  bound  on  maximum 
GUB  row  dimension  (actual  GUB  quality  may  be  better  than  this 
conservative  estimate) .   The  results  were  obtained  using  FORTRAN  H 
(Extended)  with  code  optimization. 
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3.4.   Implicit  Network  Rows 

Implicit  network  rows  are  a  set  of  rows  for  which  each 
column  has  at  most  two  non-zero  coefficients  (restricted  to  those 
rows)  and  for  which  columns  with  two  non-zero  coefficients  (in 
those  rows)  can  be  converted  by  simple    row  and  column  scaling 
such  that  the  non-zero  coefficients  have  opposite  sign.   Such 
rows  in  LP  are  commonly  called  networks  with  gains. 

Pure  network  rows  (NET)  can  be  converted  by  simple    row 
and  column  scaling  such  that  all  non-zero  coefficents  (restricted 
to  those  rows)  have  value   ±1,   and  such  that  columns  with  two 
non-zero  coefficients  (in  those  rows)  have  one   +1   and  one   -1. 
Such  rows  in  LP  are  called  pure  networks. 

Simple  row  and  column  scaling  is  restricted  such  that 
application  of  each  scale  factor  renders  an  entire  row,  or  column, 
to  the  desired  sign  (and  unit  magnitude  for  pure  NET) . 

The  problem  of  identifying  a  NET  factorization  of  maximum 
row  dimension  is  NP-hard  (Wright  [18]),  making  optimal  NET  identifi- 
cation algorithms  unattractive  in  a  practical  sense.   The  problem  of 
determining  just  the  size    of  the  maximum  NET  set  is  also  NP-hard. 
Thus,  heuristic  identification  methods  are  mandated. 

An  extension  of  GUB  heuristics  can  be  used  to  achieve 
NET  factorizations.   First,  a  GUB  set  is  determined  by  methods 
mentioned  in  Section  3.3.   Then,  a  second  GUB  set  is  found  from 
an  eligible  subset  of  remaining  rows.   The  second  GUB  set  is  con- 
ditioned such  that  its  row  members  must  possess  non-zero  coeffi- 
cients of  opposite  sign  in  each  column  for  which  the  prior  GUB 
set  has  a  non-zero  coefficient. 
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This  double-GUB  (DGUB)  identification  yields  a  bipartite 
NET  factorization.   Thus,  DGUB  heuristically  seeks  the  maximum 
embedded  transportation  or  assignment  row  factorization.   Pure 
network  equivalents  derive  from  proper  editing  of  eligible  rows. 

Generalizing  on  the  theme  of  Senju  and  Toyoda,  a 
more  general  method  has  been  developed  by  Brown  and  Wright  [10] 
for  direct  factorization  of  implicit  network  rows.   Pure 
NET  rows  are  identified  with  the  same  procedure  by  simple 
screening  of  admissible  candidate  rows. 

This  heuristic  is  designed  to  perform  a  network  factori- 
zation of  a  signed  elementary  matrix  (0,  ±1  entries  only).   It  is 
a  deletion  heuristic  which  is  feasibility  seeking.   The  measure 
of  infeasibility  at  any  point  is  a  matrix  penalty  computed  as 
the  sum  of  individual  row  penalties.   The  algorithm  is  two-phased, 
one  pass,  and  non-backtracking.   The  first  phase  yields  a  feasible 
set  of  rows,  while  the  second  phase  attempts  to  improve  the  set 
by  reincluding  rows  previously  excluded.   Each  iteration  in  Phase 
I  either  deletes  a  row  or  reflects  it  (multiplies  it  by  -1)  and 
guarantees  that  the  matrix  penalty  will  be  reduced.   Thus,  the 
number  of  iterations  in  Phase  I  is  bounded  by  the  initial  value 
of  the  matrix  penalty,  which  is  polynomially  bounded. 


Let  A  =  [  a .  .  ]   be  an  m  x  n   matrix  with   a..  =  0,±lVi,j. 


Problem:   Find  a  matrix  N  =  [n. .]   with   (m-k)   rows  and  n 

2.J 


columns  which  is  derived  from  A  by 
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1.  Deleting  k   rows  of  A  where   k  >  0, 

2.  Multiplying  zero  or  more  rows  of  A  by   -1, 
where  N  has  the  property  that  each  column  of  N  has 
at  most  one   +1   element  and  at  most  one   -1  element. 

We  wish  to  find  a  "large"   N   in  the  sense  of  containing  as  many 
rows  as  possible,  i.e.  minimize   k. 

Terminology  and  Notation: 

1.  E   is  the  set  of  row  indices  for  rows  eligible  for  inclusion 
in  N   and  is  called  the  eligible  set. 

2.  C   is  the  set  of  row  indices  for  rows  removed  from  E   in 
Phase  I  (Deletion) .   Some  rows  in  C   may  be  readmitted  to 
E   in  Phase  II.   C   is  called  the  candidate  set. 

3.  The  phrase  "reflect  row  i'  of  A"  means  to  multiply  each 
element  in  row  i '  by  -1,  i.e.   a.  ,  .  *■   -a.  .  .  V  j  . 

4.  Other  notation  will  be  defined  in  the  algorithm  itself. 

ALGORITHM: 

Phase  I  -  Deletion   of  Infeasible   Rows 

Step  0:  Initialization.       Set   E  =  {l,2,...,m},   C  =  <J>  . 

For  each  column   j   of  A  compute  the   +  penalty   (K.) 

and  the   -   penalty   (K . )   as  follows: 

K  ■  =  (   J   1    )  "  1      r  KT  =  (  I      1    )  -  1  . 

J     i€E:a. . >0  J     i€E:a. . <0 

i:  ID 

These  penalties  represent  the  number  of  excess   +1   and   -1 
elements,  respectively,  in  column  j  which  prevent  the  rows 
whose  indices  remain  in  E   from  forming  a  valid  N   matrix. 
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A  penalty  value  of   -1   for  K.(K.)   indicates  that  the 
column  does  not  contain  a   +1(-1)   element. 


Step  1:  Define   Row   Penalties .       For  every   i  €E,   compute  a  row 
penalty   (p.)   as  follows: 

p±  -   I    Kt  +  I         kT  . 

j :a.  .>0   J    j :a.  .<0   J 

This  is  simply  the  sum  of   +  penalties  for  all  columns  in 
which  row  i  has  a   +1   plus  the  sum  of   -   penalties  for 
all  columns  in  which  row  i  has  a   -1. 

Step  2:  Define   Matrix  Penalty.      Compute  the  penalty   (h)   for 
the  matrix  by  summing  the  row  penalties  as  follows: 


h  =  I      p 

i£E 


If  h  =  0,   then  go  to  Step  7.   Otherwise,  go  to  Step  3. 

Step  3:  Row   Selection.       Find  the  row   i'  €E   with  the  greatest 
penalty,  i.e. 

Find   i'  €  E   such  that   p.  ,  =  max  p.  . 

1     iCE   1 

(If  there  is  a  tie,  choose   i1   from  among  the  tied  values.) 

Compute  the  reflected  row  penalty  p.,   for   i'   as  follows: 


p.,  =    I  (kT+1)  +    I  (K.+l) 

j:ai,j>0   J       j:ai(j<0   J 
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This  would  be  the  row  penalty  for  row   i1   if  it  were  to 
be  reflected. 

Step  4:  Delete,    or   Reflect   Row. 

Case   i)   p.,  >  p.,  .   Let  E  «■  E-{i'},  C  «-  CU{i'}.   Go 

to  Step  5 . 
Case  ii)   p. ,  <  p. ,  .   Reflect  row   i1.   Go  to  Step  6. 


Step  5:  Reduce    column   penalties    as  follows: 

For  all   j   such  that   a.  ,  .  >  0,   K+  <-  K+  -  1  . 

For  all   j   such  that   a.  .  .  <  0,   K~  +■  K~  -  1  . 
J  I'D        D     D 

Go  to  Step  1. 


Step  6:  Change    column   penalties    as  follows: 

Using  the   a. , .   values  after  reflection  of  row   i1 , 

For  all   j   such  that   a.  ,  .  >  0,   K+  «-  K+  +  1   and   KT  «-  kT  -  1 
J  I'D        D     D  D     D 

For  all   j   such  that   a. .  .  <  0,   K+  «-  K+  -  1   and  K~  *■   K~  +  1 
J  I'D        D     D  3     D 

Go  to  Step  1. 

Phase  II  -  Reinclusion   of  Rows   from   C 

Step  7:  Eliminate    Conflicting  Rows.      The  rows  with  indices  in  E, 
some  possibly  reflected  from  the  original  A  matrix,  form  a 
valid  N  matrix.   However,  some  of  the  rows  removed  from  E 
and  placed  in   C   may  now  be  reincluded  in  E   if  they  do  not 
make  h  >  0.   Remove  from  C   (and  discard)  all  row  indices 
for  rows  which,  if  reincluded  in  E   in  present  or  reflected 
form,  would  make   h  >  0. 
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i.e.   Remove   i   from  C   if 

a)   3  j.   such  that   a.    >  0   and  K.   =  0  / 
1  ^Di  Di 


or  a .  .  <  0   and  K .   =  0  ; 

and 

b)   3  i.   such  that   a. .  >  0   and  kT   =  0  , 
2               ^2  ^2 

or   a.  <  0   and  K.   =  0  . 
1D2  ]2 

If  C  =  <$>,      STOP,  otherwise  go  to  Step  8. 


Step  8:  Select   Row   for  Reinolusion.      At  this  point  a  row  from 
C   may  be  reincluded  in   E.   There  are  several  possible 
schemes  for  selecting  the  row.   After  the  row  is  reincluded, 
the  column  penalties  are  adjusted.   Then  go  to  Step  7. 

No  dominating  rule  has  been  discovered  for  breaking  ties 
in  maximum  row  penalty  encountered  in  Step  3.   The  rule  used  for 
the  computational  results  presented  herein  is  to  select  the  row 
with  the  minimum  number  of  non-zero  entries  in  an  attempt  to  place 
a  larger  number  of  non-zero  entries  in  the  network  set.   Other 
possible  rules  are  "first-come,  first-served,"  maximum  number  of 
non-zero  entries,  type  of  constraint,  or  modeler  preference. 

Modifications  can  be  made  to  Step  0  to  allow  for   1) 
Matrices  including  non-  0,  ±1   entries  and/or   2)  Pre-specif ied 
network  rows.   The  modifications  are: 
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1.  E  =  {i  I  a.  .  =  0,±1   for  all   j}  , 

2.  Let   P  =  {i  |  row   i   is  prespecif ied}  , 
E  *■    E  -  P  . 

After  computation  of  K .   and  K .    find  for  all   j 

if  3i€P   such  that   a.  .  =  1   then  K+  «•  K+  +  1  , 

13  D     1 

if   3i€P   such  that   a..  =-1  then   kT  <•  K~  +  1  . 

iJ  3  3 


At  termination  of  the  algorithm,  the  rows  in  N   are  given  by 
E  U  P. 

One  easily  obtained  upper  bound  on  the  maximum  row 
dimension  of  the  network  factorization  is: 

w,  =  m-  MAX(K++KT)  . 
1       j    3      3 

This  bound  is  easily  computed  and  evidently  sharp.   It 
can  be  used  to  objectively  evaluate  the  quality  of  a  heuristically 
derived  network  factorization.   The  bound  may  also  be  used  to 
preemptively  terminate  identification  effort. 

Another,  generally  tighter  bound  has  been  developed  by 
Wright  [19]  which  is  based  on  the  reflection  and  deletion  potentials 
for  each  row  in  the  eligible  set.   Using  this  information  it 
is  possible  to  obtain  a  lower  bound  on  the  number  of  rows  which 
must  be  deleted  to  achieve  a  feasible  network  set.   The  upper 
bound  is  then: 


u~    =   m-  lower  bound  on  rows  deleted. 
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This  bound  is  also  evidently  sharp  and  is  the  bound 
used  to  compute  NET  quality  in  the  following  table. 

Table  4  displays  the  results  of  DGUB  and  NET  factoriza- 
tions of  the  benchmark  models.   Row  eligibility  is  determined  by 
the  capability  to  scale  each  row,  by  row  scaling  alone,  to  con- 
tain only   0,  ±  1   entries.   The  NET   quality    is  the  number  of  NET 
rows  found,  expressed  as  a  percentage  of  the  upper  bound  on  maxi- 
mum NET  row  dimension  given  above  (actual  NET  quality  may  be 
considerably  better  than  this  estimate) . 
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3.5.   Hidden  Network  Rows 

+ 
Hidden  network  rows   are  a  set  of  rows  which  satisfy 

NET  row  restrictions  after  full  linear  transformation  of  the 

model.   That  is,  realization  of  these  (LNET)  rows  may  require  a 

general  linear  transformation  of  the  original  model. 

The  discrimination  between  implicit    and  hidden   network 
rows  is  not  (necessarily)  in  their  use,  but  rather  in  their 
detection.   The  transformation  group  associated  with  implicit 
network  rows  involves  only    permutations  and  simple  scaling  of 
individual  rows  and  columns.   The  hidden  network  rows  require  a 
completely  general  linear  transformation  and  partial  ordering. 
Thus,  identification  of  hidden  networks  requires  significant 
computation  just  to  identify  eligible  rows,  since  any  given  row 
may  conflict  with  subsets  of  its  cohorts  after  transformation. 

This  problem  has  been  solved  for  entire   hidden  network 
factorization,  where  all  rows  are  shown  to  be  LNET  or  the  algo- 
rithm fails.   Bixby  and  Cunningham  [2]  and  Muslem  [16]  have  given 
polynomially  complex  methods  for  entire  LNET  conversion.   (The 
entire  GUB  problem  is  polynomial  as  well.) 

Strategically,  the  entire  hidden  LNET  factorization 
requires  two  steps: 

DETECTION :      necessary  conditions  for  existence  of  an  entire 

LNET  factorization  must  be  established,  and 
SCALING:  a  linear  transformation  to  achieve  the  NET 

structure  must  be  determined,  if  one  exists. 


+ 
We  have  coopted  the  term  hidden    from  Bixby  [1],  but  his  defini- 
tion may  not  superficially  appear  to  be  equivalent. 
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Cunningham  and  Bixby  attempt  detection,  followed  by 
scaling.   Musalem  tries  scaling,  then  detection.   This  is  a  cru- 
cial difference  between  methods,  since  problems  which  can  not  be 
completely  NET  factorized  may  fail  in  either  step. 

Briefly,  Cunningham  and  Bixby  detect   by  showing  that 
the  incidence  matrix  of  the  model  rows  can  be  converted  to  a 
graphic  matroid.   They  employ  a  method  by  Tutte  (see  references 
of  Bixby  and  Cunningham.   Given  success,  the  graphic  record 
of  the  detection  is  used  to  attempt  to  scale    the  model  to  NET,  or 
to  show  that  no  such  scaling  exists. 

Musalem  scales    the  model  to  a   ±1   matrix,  and  then  uses 
a  method  by  Iri  (see  references  of  Musalem)  to  build  a  tree, 
edge  by  edge,  which  reveals  the  partial  ordering  coincident  with 
entire  hidden  LNET  factorization. 

Both  methods  are  polyncmially  complex.   However, 
entire  LNET  factorization  is  relatively  expensive  by  either  method 
in  that  quite  a  large  amount  of  real  arithmetic  and  logic  is 
required.   Underlying  data  structures  have  not  been  suggested 
for  either  method.   Both  methods  fail  if  complete  LNET  factoriza- 
tion is  impossible,  and  neither  leaves  the  investigator  with  much 
information  useful  in  salvaging  a  partial  LNET  factorization.   We 
conjecture  that  risk  of  preemptive  failure  narrowly  favors  the 
Musalem  approach,  since  he  defers  the  relatively  involved 
detection  step. 

Locating  a  hidden  LNET  factorization  of  maximal    row 
dimension  has  been  suggested  by  Bixby  [1]  and  by  Musalem, 
but  no  concrete  method  is  given  and  no  computational  testing  is 


32 


reported.  Evidently,  the  maximum  LNET  problem  is  NP-hard,  and 
its  maximal  relaxation  remains  unsolved  in  the  practical  sense 
of  this  report. 
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4.   EXTENSIONS  TO  MIXED  INTEGER  AND  NONLINEAR  MODELS 

Mixed  integer  (MIP)  and  nonlinear  (NLP)  optimization 
models  present  additional  challenges,  especially  at  large  scale. 
Our  interest  in  the  detection  of  structural  redundancy  is  inten- 
sified since  general  purpose  algorithms  for  (MIP)  and  (NLP)  nor- 
mally operate  by  solving  sequences   of  many    embedded  LP  models. 
This  provides  added  impetus  to  the  analysis  of  problems  prior  to 
their  actual  solution,  and  economically  justifies  some  additional 
initial  investment  in  problem  analysis. 

4.1.   Mixed  Integer  Extensions 

The  structural  analysis  presented  for  LP  is  also  appli- 
cable to  MIP.   Since  most  real-world  models  and  many  commercial 
optimization  systems  have  only  binary    variable  capability,  our 
analysis  addresses  binary  variables  exclusively.   Thus,  binary 
expansion  of  integer  variables  is  a  prerequisite.   We  also 
assume  that  reductions  requiring  scaling  of  binary  columns  are 
inadmissable . 

When  a  reduction  tightens  a  bound  for  a  discrete  variable 
the  bound  is  rounded  to  the  nearest  integer  (down  for   u.   and  up 
for  I.)  .   Any  tightening  of  bounds  for  a  binary  variable  immedi- 
ately results  in  a  fixed  column  or  an  inconsistent  column.   The 
reductions  for  fixed  columns,  vacuous  columns,  inconsistent 
columns,  singleton  constraints,  vacuous  constraints,  constraints 
that  fix  variables,  and  redundant  constraints  are  applied  exactly 
as  described  for  LP. 

The  treatment  of  doubleton  equations  requires  special 
consideration  to  identify  inconsistent  constraints. 
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A  doubleton  equation  with  both  columns  binary  has 
either  one  solution,  two  solutions  or  no  solutions.   All  four 
possible  solutions  (0,0),  (1,0),  (0,1)  and  (1,1)  are  tried. 

If  only  one  solves  the  equation,  the  binary  variables 
are  fixed  at  these  values  and  the  constraint  is  removed  (the 
test  for  constraints  that  fix  variables  will  also  discover  the 
equation  with  a  single  solution  and  accomplish  the  same  reduction) 

For  the  case  with  two  solutions  there  can  only  be  two 

situations:   consider  the  doubleton  as   a.,X,  +  a.0X~  =  b.   with 

ll  1    i2  2     i 

a.,  ?   0   and   a-2  ^  0.   Then  if   (0,0)   and   (1,1)   can  both 
solve  the  equation,  this  implies  that  b  =  0   and   a.,  -  -a..~. 
If   (0,1)   and   (1,0)   are  solutions,  this  implies  that 
a.,  =  a. 2  =  b.   Both  cases  are  treated  correctly  by  the  trans- 
formation described  for  continuous  variables. 

If  there  are  no  solutions,  the  constraint  is  designated 
as  inconsistent. 

For  a  doubleton  equation  with  one  continuous  and  one 
binary  variable,  the  transformation  described  for  continuous 
variables  is  used,  but  it  must  be  the  continuous  variable  that 
is  eliminated. 

The  reductions  to  eliminate  singleton  columns  and  to 
designate  a  free  column  equation  may  not  be  applied  to  binary 
variables.   This  is  ensured  by  marking  the  binary  columns  as 
ineligible  for  these  reductions.   Note  that  since  a  binary  vari- 
able can  never  be  designated  as  a  free  column,  binary  variables 
cannot  be  eliminated  as  a  variable  in  a  free  column  constraint 
but  may  be  among  the  other  columns  in  such  a  constraint. 
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The  redundant  bounds  test  may  be  used  to  tighten  bounds 
on  binary  variables  and  thus  fix  them,  or  show  that  the  MIP  has 
no  feasible  solution. 
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4.2.   Nonlinear  Extensions 

Large-scale  nonlinear  optimization,  though  not  yet  in 
wide  use,  can  benefit  from  the  analysis  techniques  given  here 
for  LP,  and  demands  some  additional  special  treatment. 

We  have  experience  with  only  two  large-scale,  general- 
purpose  optimization  systems  with  full  nonlinear  capability:   our 
own  X-system  and  MINOS  /  Augmented  [17].   Both  of  these  systems 
can  accept  linear  problem  features  and  labels  in  MPS  format  and 
nonlinear  terms  from  function  generators.   Both  systems  can  also 
employ  several  alternate  problem  generation  interface  standards. 

These  systems  are  each  designed  to  exploit  any  linearity 
or  near-linearity  in  the  NLP.   Given  a  starting  solution,  it  is 
of  no  little  interest  to  analyze  the  linear  portion  of  the  NLP 
which  will,  after  all,  be  solved  many  times--the  reason  that 
we  support  all   LP  features  for  NLP.    Also,  any  local  lineariza- 
tion of  the  NLP  is  subject  to  analysis. 

However,  it  has  been  our  misfortune  to  have  repeatedly 
discovered  that  NLP  presents  us  with  unique  structural  curiosities. 

We  refer  to  the  foremost  among  these  as  function   coordi- 
nation.      There  are  myriad  opportunities  with  NLP  to  unwittingly 
introduce  discontinuities  and  miscellaneous  unruliness  in  functions 
and  derivatives.   Whether  by  programming  error,  mathematical 
blunder  or  numerical  difficulty,  these  errors  inflict  great 
vexation  and  expense. 

Detection  of  such  difficulties  is  quite  challenging 
since,  unlike  LP,  procedures  and  data  are  used  to  express  the 
problem  at  hand.   As  a  bare  minimum,  we  employ  a  preemptive  analysis 
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module  that  acts  as  a  complete  surrogate  for  the  optimizer, 
employing  standard  interface  conventions  and  exercising  all 
functions  and  data. 

The  starting  solution  and  scaling  parameters  are  used 
to  check  analytic  gradients  (if  supplied)  with  numerical  differ- 
ence approximations.   Approximation  of  functions  is  then  attempted 
to  reveal  behavior  local  to  the  initial  solution  such  as  apparent 
convexity  and  degree  of  nonlinearity.   Optionally,  the  first  step 
of  the  algorithm  is  simulated  and  the  same  analyses  performed. 

From  initial  results  algorithm  tolerances  may  be  changed, 
programming  errors  detected,  and  so  forth,  until  acceptable  model 
behavior  is  observed. 

In  some  cases,  suspicious  functions  may  be  evaluated 
at  column  bounds  to  see  if  numerical  arithmetic  faults  occur. 
Some  models  require  construction  and  maintenance  of  a  trust   region 
for  the  approximations  implied  by  the  NLP  algorithm,  and  prior 
analysis  is  absolutely  essential  in  these  cases. 

Structural  analysis  of  MIP  and  NLP  can  frequently — 
even  repeatedly — presage  outright  failure  of  the  solution  algo- 
rithms to  be  employed.   For  these  models,  the  effects  of  struc- 
tural redundance  can  be  far  more  significant  than  for  simple  LP. 
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5.   CONCLUSION 

The  techniques  reported  here  have  been  used  with  great 
success  on  a  wide  variety  of  large  LP  (MIP)  models.   The  context 
of  this  research  is  somewhat  atypical  in  that  the  models  which 
we  work  with  are  often  sent  to  us  for  analysis  and  solution  pre- 
cisely because  they  have  already  failed  elsewhere.   In  these 
cases,  our  motives  are  to  quickly  diagnose  suspected  trouble 
before  optimization/  prescribe  remedies,  and  perform  the  actual 
optimization  reliably  and  efficiently. 

This  has  undoubtedly  biased  our  view  of  structural  detec- 
tion methods.   Practical  considerations  arising  from  turnaround 

deadlines  and  the  specific  advantages  of  our  own  optimization 

f 
system  (Brown  and  Graves  [8])  have  colored  our  judgment.   Many 

provocative  suggestions  for  further  research  have  not  been  pursued, 
either  due  to  lack  of  opportunity,  to  poor  intuition,  or  to  simple 
economics.   Whether  or  not  by  equivalent  prejudice,  Krabek  [14] 
reports  some  similar  methods  for  detecting  redundancy  in  large- 
scale  MIP. 

Various  commercial  optimization  systems  support  "CRASH," 
"REDUCE,"  and  other  operators  which  implement  some  of  these  reduc- 
tions automatically  during  LP  solution.   These  systems  are  not 
reviewed  here.   We  stress  the  value  of  structural  analysis 


The  X-system    (XS)  differs  in  many  ways  from  classical  large- 
scale  mathematical  programming  systems;  it  simultaneously  sup- 
ports simple  and  generalized  upper  bounds,  general  basis  fac- 
torization, MIP,  nonlinear,  and  decomposition  features.   In 
addition,  the  fundamental  LP  algorithm  has  been  enhanced  to 
intrinsically  incorporate  elastic   range  restrictions.   XS  is 
particularly  suited  for  solution  in  limited  time  of  large  models 
with  complicating  features. 
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techniques  as  stand-alone    tools,  rather  than  as  exclusive 
features  of  actual  LP-solution  algorithms. 

A  great  deal  of  insight  has  been  gained  from  these  experi- 
ments.  The  cost  of  analysis  is  truly  insignificant  relative  to 
the  information  and  solution  efficiency  gained  thereby.   Revela- 
tions have  ranged  from  outright  rejection  of  absurd  formulations, 
to  subtle  inferences  on  the  project  management  and  inter-personal 
relations  among  model  proponents.   Very  few  models  fail  to  reveal 
some  totally  unsuspected  structural  curiosity.   Indeed,  it  is 
often  some  small  aberration  that  proves  most  revealing.   Sometimes, 
the  combined  effects  of  several  minor  features  collectively  con- 
tribute to  a  discovery  of  significant  model  attributes. 

Our  general  operational  guideline  has  been  to  avoid  heavy 
computational  investment  in  model  analysis.   Rather,  highly 
efficient  methods  are  used  repeatedly   on  variations  of  each  model. 
Manual  and  intuitive    analysis  of  these  results  usually  reveal 
much  more  than  could  be  reasonably  expected  from  any  totally  auto- 
mated method  of  exponential  complexity.   After  all,  just  the  names 
of  rows  and  columns  can  be  expected  to  reveal  a  great  deal  about 
the  model,  but  exploiting  this  mathematically  virtually  defies 
automation  in  any  general  sense;  interactive  analysis  of  large- 
scale  models  is  uncompromisingly  challenging  in  a  technical  sense 
and  equally  rewarding. 

Large  degrees  of  structural  redundancy  are  routinely  found 
as  intrinsic  features  in  real-life  models.  However,  we  feel  that 
it  is  an  abominable  practice  to  proselytize  in  favor  of  some  par- 
ticular model  structure  at  the  expense  of  model  realism  or  common 
sense. 
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For  instance,  network  models  have  recently  received 
unprecedented  attention  in  the  literature.   The  implication  has 
often  been  that  since  networks  are  usually  found  in  models,  net- 
works should  be  used  as  the  exclusive  model.   This  is,  of  course, 
patent  nonsense,  smacking  of  a  solution  in  search  of  a  problem. 
An  analyst  should  view  intrinsic  redundancy  as  an  interesting 
feature  of  models,  rather  than  forcing  models  to  exhibit  minimal 
redundancy,  or  requiring  that  they  follow  some  particular  struc- 
tural pattern. 

As  for  automating  the  discovery  of  all    redundancy  in  a 
model,  this  exercise  seems  to  be  almost  exclusively  academic  with 
large-scale,  real-life  LP  projects.   In  those  rare  cases  for  which 
such  extensive  analysis  is  justified,  we  suggest  a  straightforward 
view  and  a  frontal  attack  with  an  imbedded  LP  optimizer. 
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